Image synthesis system for a vehicle and the manufacturing method thereof

ABSTRACT

The present invention discloses an image synthesis system for a vehicle to provide the driver with a downward-facing image of the car&#39;s 360° surrounding view. The system includes: a first camera, which is used to shoot a first image of the periphery of the vehicle; a second camera, which is used to shoot a second image of the periphery of said vehicle, wherein the second image and the first image have an overlap region; an image processing device comprising a defining component and a synthesis component, which is used to synthesize the first image and the second image and output a third image; a display device, which is used to display the third image.

FIELD OF THE INVENTION

The present invention relates to an image synthesis system for avehicle. More particularly, it relates to a system which is disposed ona vehicle to provide the driver a downward-facing image of the vehicle's360° surrounding view to monitor the periphery of the vehicle.

BACKGROUND OF THE INVENTION

Since each vehicle has different design, when a driver drives differentvehicles, he/she will have different blind spots, which threatens thedriver's safety, especially to drive in a more complicated environmentsuch as narrow alley and garage. The conventional way to solve thisproblem is to mount a rear-view camera on the back of the vehicle forthe driver being able to see the back while backing off. But the driverstill needs to pay attention to the rear-view mirrors of the sides andthe front area. It is difficult for the driver to take care of each sideof the vehicle at the same time, and the accidents happen easily.

U.S. Pat. No. 7,139,412 applied by Nissan disclosed a car surveillancesystem which comprises: mounting several cameras on a vehicle, shootingthe images of the periphery of said vehicle, arranging all the imagesinto one image according to their positions around the vehicle, anddisplaying said image on a screen installed in the car. However, ifthere is a 3D object presenting on the borders of the images, the edgesof the object in one image will misalign to the edges in the otherimage, so that the systems can't provide accurate visual guidance forthe driver.

In view of the disadvantages, the present invention provides an imagesynthesis system for a vehicle and the manufacturing method thereofwhich comprises: shooting the images of the periphery of a vehicle bythe cameras disposed on said vehicle, synthesizing the images into adownward-facing image, make the stitching of two conjunct images smoothand seam-less, and displaying a high quality synthesized image on adisplay device, to achieve a better monitoring effect.

SUMMARY OF THE INVENTION

The present invention provides an image synthesis system for a vehicleand the manufacturing method thereof based on the problems stated above.

One object of the present invention is to provide an image synthesissystem for a vehicle, capable of providing a high quality synthesizedimage of 360° surrounding view of the vehicle and achieving a bettermonitoring effect.

Another object of the present invention is to provide an image synthesissystem for a vehicle, which applies the techniques in computer sciencecategory, capable of making the stitching of two conjunct images smoothand seam-less.

Based on the objects stated above, the image synthesis system for avehicle of the present invention comprises: a first camera, a secondcamera, an image processing device, and a display device. Said imageprocessing device comprises: a defining component, a synthesiscomponent, a transformation component, a seam registration component,and a deformation component. The first camera is disposed on a vehicleand shooting a first image of the periphery of the vehicle; the secondcamera is disposed on the vehicle and shooting a second image of theperiphery of the vehicle, wherein the second image and the first imagehave an overlap region; the transformation component of the imageprocessing device is for transforming the first image and second imageinto a downward-facing image, and the synthesis component is forsynthesizing the first image and the second image and outputting a thirdimage. To generate the third image, first, the defining componentdefines the first image into a plurality of first points and defines thesecond image into a plurality of second points, wherein each point has aresidual error. When the first point overlaps to the second point, thepoint with lower residual error will be shown on the third image. Thethird image has an optimal stitching seam between the synthesized firstimage and second image, wherein at least one object presents on the seamand the edges of the object in the first image misaligned to the edgesin the second image. To solve this problem, the seam registrationcomponent of the image processing device is provided to align the edgesof the object on the seam. The image processing device further comprisesthe deformation component to propagate the alignment of the edges of theobject to the rest of the first image and the second image, and tooutput a complete third image.

The present invention further provides an image synthesis method for avehicle which comprises following steps: shooting a fist image of theperiphery of a vehicle, shooting a second image of the periphery of saidvehicle, transforming the first image and second image into adownward-facing image, processing the image to synthesize the firstimage and the second image and to output a third image, and displayingthe third image. Processing the image further comprises: defining thefirst image into a plurality of first points and defining the secondimage into a plurality of second points each has a residual error,synthesizing the first image and the second image and outputting a thirdimage, aligning the edges of the object on the seam, and deforming thefirst image and the second image by propagating the alignment of theedges of the object to the rest of the first image and the second image.Below is the process of the steps. First, using the first camera toshoot a first image of the periphery of the vehicle, and using thesecond camera to shoot the second image of the periphery of the vehicle,wherein the second image has an overlap region with the first image;transforming the first image and second image into a downward-facingimage; then synthesizing the first image and the second image andoutputting a third image. To generate the third image, first, define thefirst image into a plurality of first points and defining the secondimage into a plurality of second points, wherein each point has aresidual error. When the first point overlaps to the second point, thepoint with lower residual error will be shown on the third image. Thethird image has an optimal stitching seam between the synthesized firstimage and second image, wherein at least one object presents on the seamand the edges of the object in the first image misaligned to the edgesin the second image. Therefore, the step to align the edges of theobject on the two sides of the seam is required. And then the next is topropagate the alignment of the edges of the object to the rest of thefirst image and the second image and to generate a complete third image.

The present invention further provides an image synthesis device forsynthesizing a first image and a second image, wherein the first imageis taken by a first camera disposed on a vehicle, and the second imageis taken by a second camera disposed on the vehicle, and the secondimage and the first image have an overlap region, the image synthesisdevice comprising: a defining component for defining the first image andsecond image into a plurality of points, each point has a residualerror; a synthesis component for synthesizing the first image and thesecond image and outputting a third image, wherein when the first pointoverlaps to the second point, the point with lower residual error willbe shown on the third image. The image synthesis device further includesa transformation component for transforming the first image and thesecond image into a downward-facing image. The third image has at leasta seam between the synthesized first image and second image, wherein atleast one object presents on the seam and the edges of the object in thefirst image misaligned to the edges in the second image. Therefore, theimage synthesis device further includes a seam registration componentfor aligning the edges of the object of the two sides of said seam. Theimage synthesis device further includes a deformation component forpropagating the alignment to the rest of the first image and the secondimage and outputting a complete third image.

The present invention further provides an image synthesis method forsynthesizing a first image and a second image, wherein the first imageis taken by a first camera disposed on a vehicle, and the second imageis taken by a second camera disposed on the vehicle, and the secondimage and the first image have an overlap region, the image synthesismethod comprising: defining the first image and the second image into aplurality of points and each point has a residual error; synthesizingthe first image and the second image and outputting a third image,wherein when the first point overlaps to the second point, only thepoint with lower residual error will be shown on the third image. Theimage synthesis method further includes transforming the first image andsecond image into a downward-facing image. The third image has anoptimal stitching seam between the synthesized first image and secondimage, wherein at least one object presents on the seam and the edges ofthe object in the first image misaligned to the edges in the secondimage. Therefore the method comprises aligning the edges of the objecton the seam. The image synthesis method further comprises deforming thefirst image and the second image by propagating the alignment of seam tothe rest of the first image and the second image and outputting acomplete third image.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 a block diagram of an embodiment of the image synthesis systemfor a vehicle in accordance with the present invention;

FIG. 2 illustrates the first image taken by the first camera and thesecond image taken by the second camera and the overlap region of thetwo images and the seam;

FIG. 3 illustrates the third image synthesized by the first image andthe second image;

FIG. 4 a flow chart of an embodiment of the image synthesis method for avehicle in accordance with the present invention;

FIG. 5 a detailed flow chart of an embodiment of the image synthesismethod for a vehicle in accordance with the present invention;

FIG. 6 illustrates the seam and the data sequences A and B coming fromdifferent resources along the seam;

FIG. 7 illustrates the wrapping path starts from w₁=(1,1) and ends atw_(m)=(n,n);

FIG. 8 a block diagram of an embodiment of the image synthesis device inaccordance with the present invention; and

FIG. 9 a flow chart of an embodiment of the image synthesis method inaccordance with the present invention.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Refer to FIG. 1, FIG. 2, and FIG. 3. In accordance with one embodimentof the present invention an image synthesis system for vehicle 100,comprises: a first camera 110, a second camera 120, an image processingdevice 130, and a display device 140. Said image processing devicecomprises: a defining component 131, a synthesis component 133, atransformation component 135, and a seam registration component 137, anda deformation component 139. The first camera 110 is disposed on avehicle and shooting a first image 210 of the periphery of the vehicle;the second camera 120 is disposed on the vehicle and shooting a secondimage 230 of the periphery of the vehicle, wherein the second image andthe first image have an overlap region 220; the transformation component135 of the image processing device is for transforming the first imageand second image into a downward-facing image, and then using thesynthesis component 133 to synthesize the first image and the secondimage and output a third image 300. To generate the third image, first,the defining component 131 defines the first image 210 into a pluralityof first points 211 and defines the second image 230 into a plurality ofsecond points 231, wherein each point has a residual error. When thefirst point 211 overlaps to the second point 231, the point with lowerresidual error will be shown on the third image. Therefore, the thirdimage has at least one seam 221 between the synthesized first image andsecond image, wherein at least one object presents on the seam and theedges of the object in the first image misaligned to the edges in thesecond image. Therefore, the seam registration component 137 of theimage processing device is required to align the edges of the object onthe seam. Furthermore, the image processing device includes thedeformation component 139 to propagate the alignment of the edges of theobject to the rest of the first image and the second image, and tooutput a complete third image.

Refer to FIG. 4, which illustrates an image synthesis method for avehicle 400 in accordance with one embodiment of the present invention,comprises following steps: shooting a fist image of the periphery of avehicle 410, shooting a second image of the periphery of said vehicle420, transforming the first image and second image into adownward-facing image 430, processing the image to synthesize the firstimage and the second image and to output a third image 440, anddisplaying the third image 450. Processing the image 440 furthercomprises: defining the first image into a plurality of first points anddefining the second image into a plurality of second points each has aresidual error 442; synthesizing the first image and the second imageand outputting a third image 444, wherein when the first point overlapto the second point, the point wither lower residual error will be shownon the third image; aligning the edges of the object presenting on theseam 446; and deforming the first image and the second image bypropagating the alignment of the edges of the object to the rest of thefirst image and the second image 448.

FIG. 5 illustrates the detailed steps of this embodiment, and theexplanation is made by referring both FIGS. 4 and 5. Refer to FIG. 5.First, the step 502 is for calibrating the cameras, and exacting thecameras' intrinsic parameters and extrinsic parameters (step 504). Next,set up the calibrated cameras on the vehicle (step 506) to take theimages of the periphery of the vehicle (step 508). Step 508 refers tosteps 410 and 420 in FIG. 4. Use the camera parameters got from step 504and correct distortion of the taken images from step 508 (step 510) toget a perspective projection image (step 512). Shoot a ground image fromthe top of the vehicle beforehand (step 514). Register the images fromstep 508 and image from step 514 (step 516), and then combine the imagesfrom step 508 to a downward-facing image (step 518). The steps 514-518refer to the step 430 in FIG. 4. The two images taken from conjunctcameras have an overlap region, wherein an optimal stitching seam isable to be found. To do so, first, define the two images into aplurality of points (step 442), wherein each point has a residual error.Select a seam which is consisted of points with lowest residual error(step 221), and then stitch the two images to output a third image (step444).

Following the steps stated above already can stitch multiple partialscene images of the periphery of the vehicle and output a completedownward-facing surrounding view image. Since the method has beenapplied is Homography Matrix, which is used for image registration on aplanar surface. Therefore, if there is a 3D object presenting on thestitching seam, the object's edges of the two sides of the seam willmisalign. To solve this problem, image registration is required (step446). In this embodiment, dynamic image warping (DIW) is applied. Referto FIG. 6. Suppose there are two images Ia and Ib stitched together onthe seam Sab. Since the seam lies in the overlap region of Ia and Ib,two series of pixel data A and B are obtained which come from Ia and Ibalong Sab

A=a₁,a₂, . . . ,a_(i), . . . ,a_(n)   (1)

B=b₁,b₂, . . . ,b_(j), . . . ,b_(n)   (2)

Where n is the length of Sab in pixel. Refer to FIG. 7. To align twosequences using DTW, an n×n matrix is constructed where the (i, j)element of the matrix contains the difference value d(ai, bj) betweenthe two nodes ai and bj. A warping path W is a set of matrix elementsthat describes a mapping between A and B.

W=w ₁ , w ₂ , . . . ,w _(k) , . . . ,w _(m) n≦m≦2n−1   (3)

The warping path is typically subject to the following constraints.

(1) Boundary Condition

The warp path must start in w₁=(1,1) and finish in w_(m)=(n,n). Thesetwo element is the opposite corner elements of the matrix. In orderword, the starting nodes of both sequences must be registered together,same as the ending nodes.

(2) Continuity

Given w_(k)=(p,q) and w_(k−1)=(p′,q′) where p−p′≦1 and q−q′≦1. Thisrestricts the warp path to step forward only into the adjacent element,and diagonally adjacent elements are also included.

(3) Monotonicity

Given w_(k)=(p,q) and w_(k−1)=(p′, q′) where p−p′≧0 and q−q′≧0. Thisforces the points in W to be monotonically spaced in spatial domain.

There are exponentially many warping paths that satisfy the aboveconstraints. However, we are interested only in the warping path thatminimizes the warping cost C. Formula (4) is used to calculate the C.

$\begin{matrix}{{C\left( {i,j} \right)} = {{D\left( {a_{i},b_{j}} \right)} + {H\left( {a_{i},b_{j}} \right)} + {\min \left\{ {{\begin{matrix}{{C\left( {i,{j - 1}} \right)} + p_{v}} \\{C\left( {{i - 1},{j - 1}} \right)} \\{{C\left( {{i - 1},j} \right)} + p_{h}}\end{matrix}{C\left( {1,1} \right)}} = 0} \right.}}} & (4)\end{matrix}$

where p_(h) and p_(v) are the standard deviation of data sequence A andB, and D(a_(i), b_(j)) is the difference measurement between data pointa_(i) and b_(j). This difference is composed by two terms, thedifference of absolute values and the difference of first derivativeangles, can be written as

D(a _(i) ,b _(j))=d(a _(i) ,b _(j))+d′(a _(i) ,b _(j))   (5)

where

d(a _(i) ,b _(j))=(a _(i) ,b _(j))²   (6)

d′(a _(i) ,b _(j))=arc tan(a _(i)′)−arc tan(b _(j)′)   (7)

where

$\begin{matrix}{q_{i}^{\prime} = {\frac{\left( {q_{i} - q_{i - 1}} \right) + \left( \frac{q_{i + 1} - q_{i - 1}}{2} \right)}{2}.}} & (8)\end{matrix}$

Formula (8) is from Derivative Dynamic Time Warping (DDTW) proposed byKeogh et al. which is the improvement of Dynamic Time Warping (DTW), andis to consider the shape-level of pixel data for image registration.Formula (7) inherits DDTW. The results from Formulas (6) and (7) arerequired to be normalized to 0˜1. Otherwise, the history of the seamregistrations are also taken into consideration H(a_(i),b_(j)) can bewritten as:

H(a _(i) ,b _(j))=s((i−h _(b)(f,j))²+(j−h _(a)(f,i))²)   (9)

where h_(a)(f,i) is the average index of the corresponding point in dataseries B to the j-th point in data series A in f previous frames.h_(b)(f,j) is the average index of the corresponding point in dataseries A to the j-th point in data series B in f previous frames, and sis a scalar. As s increasing, the result of Formula (9) in current framehas a bias to the history of the seam registration.

Please refer to FIG. 4 and FIG. 5. Once the alignment of the seams ofthe adjacent images is done, next step is to propagate this alignmentresult to the rest of the images. This is a typical case of the imagedeformation. In general, image deformation is accomplished by thetransformation function T(x). The interpolation transformation functionT(x) based on point-landmarks must map the landmarkp_(i)=(p_(ix),p_(iy))ε R² in the source image to its landmarkq_(i)=(q_(ix),q_(iy))ε R² in the target image, can be written as:

T(p _(i))=q _(i) , i=1, . . . ,n,   (10)

where n is the number of the landmarks. The transformation functions intwo coordinates are calculated separately.

$\begin{matrix}{\begin{matrix}{{T\left( p_{i} \right)} = \left( {{t_{x}\left( p_{ix} \right)},{t_{y}\left( p_{iy} \right)}} \right)} \\{{= \left( {q_{ix},q_{iy}} \right)},}\end{matrix}{{i = 1},\ldots \mspace{11mu},n}} & (11)\end{matrix}$

where t_(x)(.) and t_(y) (.) are the transformation functions in x and ycoordinates, respectively. In radial basis function approach, thetransformation is composed by two terms as follow:

t(x)=R _(s)(x)+L _(s)(x)   (12)

where R(x) is the non-linear term which consists of the weighted RBFs,and L(x) is the linear term which consist of m bases of polynomials upto degree d.

$\begin{matrix}{{R_{s}(x)} = {\sum\limits_{i = 1}^{n}{\alpha_{i}{R\left( {{x - p_{i}}} \right)}}}} & (13) \\{{L_{s}(x)} = {\sum\limits_{j = 1}^{m}{\beta_{j}{L_{j}(x)}}}} & (14)\end{matrix}$

where R (∥x−p_(i)∥) is the radial basis function centered aroundlandmark p_(i), and its value only depend on the Euclidean distance fromx to p_(i). α_(i) and β_(j) are coefficients. In order to preserve theoverall smoothness as much as possible, the coefficient α_(i) istypically subject to the following constraint:

$\begin{matrix}{{{\sum\limits_{i = 1}^{n}{\alpha_{i}{L_{j}\left( p_{i} \right)}}} = 0},{j = 1},\ldots \mspace{11mu},m} & (15)\end{matrix}$

A linear combination of the coefficients α=[α₁ . . . α_(n)]^(T) andβ=[β₁ . . . β_(m)]^(T) can be derived from the above equations andwritten as follow:

$\begin{matrix}{{{\begin{bmatrix}K & P \\P^{T} & 0\end{bmatrix}\begin{bmatrix}\alpha \\\beta\end{bmatrix}} = \begin{bmatrix}q \\0\end{bmatrix}},} & (16)\end{matrix}$

where K is the n×n sub-matrix which consists of k_(ij)=R(∥P_(i)−P_(j)∥),and P is the n×m sub-matrix which consists of P_(ij)=L_(j)(P_(i)). q=[q₁. . . q_(n)]^(T) is the target sub-matrix.

Type of RBF will affect the registration result. In this embodiment,Wendland functions are adopted and can be written as:

ψ_(d,k)(r)=I ^(k)(1−r)₊ ^(└d/2┘+k+1)(r),   (17)

where

$\begin{matrix}{\left( {1 - r} \right)_{+}^{v} = {{\psi (r)} = \left\{ \begin{matrix}{\left( {1 - r} \right)^{v},} & {0 \leq r < 1} \\{0,} & {r \geq 1}\end{matrix} \right.}} & (18) \\{{{I\; {\psi (r)}} = {\int_{r}^{\infty}{t\; {\psi (t)}{t}}}},{r \geq 0}} & (19)\end{matrix}$

Fornefett et al. proposed the method of compact support, where they usefunctions of Wendland as RBFs for elastic registration. The functions ofWendland have compact support by scaling the weight of r and can bewritten as follow:

$\begin{matrix}{{{\psi_{d,k,s}(r)} = {\psi_{d,k}\left( \frac{r}{s} \right)}},} & (20)\end{matrix}$

where s is the length of the spatial support. This approach limits thelocality of each landmark in a circle with radius s.

The images to be deformed in this application are 2D images. To createthe smooth deformation, ψ_(2,1)(r) is adopted as RBF as follow:

ψ_(2,1)(r)=(1−r)₊ ⁴(4r+1).   (21)

After finishing the deformation (step 524), all edges crossing the seamscould be stitched together on the seams. But the seams may be visibleand obvious, since the images being stitched might not be taken with thesame exposure. In order to compensate exposure difference, the bias andgain model is adopted to adjust the global exposure:

I′ _(i) =α _(i) I+β _(i),   (22)

where β is bias and α is gain. The bias and gain for each image can beobtained in the least squares formulation:

$\begin{matrix}{{E_{i} = {\sum\limits_{j}{\sum\limits_{p}\left\lbrack {{\alpha_{i}{I_{i}\left( {H_{ir}p} \right)}} + \beta_{i} - {I_{j}\left( {H_{jr}p} \right)}} \right\rbrack^{2}}}},} & (23)\end{matrix}$

where image I_(j) is the adjacent image of image I_(i), and p is theimage point in the overlap of image I_(i) and image I_(j). In thisapproach, the images can be adjusted into similar exposure. But theseams may be still visible. Hence, an image fusion method based on theweighted blending to smooth the seams is proposed. The residual error ofcamera calibration is used as the weight to blend the source images ofthe same pixels in the final composite. Though the weights of both imagesources are equal along the seam, the weighting function is notcontinuous between overlapping region and non-overlapping region.Further take the minimum distance to images boundary into weightingfunction, the proposed blending function can be formulated as

$\begin{matrix}{{{I(p)} = \frac{\sum\limits_{i}{{E_{i}\left( {H_{ir}p} \right)}{B\left( {H_{ir}p} \right)}{I_{i}\left( {H_{ir}p} \right)}}}{\sum\limits_{i}{{E_{i}\left( {H_{ir}p} \right)}{B\left( {H_{ir}p} \right)}}}},} & (24) \\{{{B\left( {u,v} \right)} = {\left( {1 - {{\frac{2u}{width} - 1}}} \right)\left( {1 - {{\frac{2v}{height} - 1}}} \right)}},} & (25)\end{matrix}$

where I(.) is the image of the final composite, E_(i)(.) is thecalibration error function of the camera i, width×height is theresolution of images, and B (.,.) is the weighting function with itsvalue 1 on the image center and 0 on the image boundary. After imagefusion (step 526), a seam-free synthesized image is output (step 528).Finally, display the image (step 530), which refers to the step 450 inFIG. 4.

Refer to FIG. 8. In accordance with one embodiment of the presentinvention, an image synthesis device for synthesizing a first image 210and a second image 230, wherein the first image is taken by a firstcamera disposed on a vehicle, and the second image is taken by a secondcamera disposed on the vehicle, and the second image and the first imagehave an overlap region 220, the image synthesis device comprising: adefining component 131 for defining the first image 210 and second image230 into a plurality of points, each point has a residual error; asynthesis component 133 for synthesizing the first image and the secondimage and outputting a third image, wherein when the first pointoverlaps to the second point, the point with lower residual error willbe shown on the third image 300. The image synthesis device furtherincludes a transformation component 135 for transforming the first imageand the second image into a downward-facing image. The third image 300has at least a seam 221 between the synthesized first image and secondimage, wherein at least one object presents on the seam and the edges ofthe object in the first image misaligned to the edges in the secondimage. Therefore, the image synthesis device further includes a seamregistration component 137 for aligning the edges of the object of thetwo sides of said seam. The image synthesis device further includes adeformation component 139 for propagating the alignment to the rest ofthe first image and the second image and outputting a complete thirdimage.

Refer to FIG. 9. In accordance with one embodiment of the presentinvention, an image synthesis method for synthesizing a first image 210and a second image 230, wherein the first image is taken by a firstcamera disposed on a vehicle, and the second image is taken by a secondcamera disposed on the vehicle, and the second image and the first imagehave an overlap region 220, the image synthesis method comprising:transforming the first image and second image into a downward-facingimage 430; defining the first image and the second image into aplurality of points and each point has a residual error 442;synthesizing the first image and the second image and outputting a thirdimage 444, wherein when the first point 211 overlaps to the second point231 only the point which has lower residual error will be shown in thethird image 300; aligning the edges of 3D objects on the seam 446; anddeformation 448. Using the cameras disposed on a vehicle to get thefirst image 210 and second image 230, and then register the two imageswith the ground image taken beforehand. Synthesize the images into adownward-facing image (step 430). The images taken from two conjunctcameras will have an overlap region, and there is at least one seam(221) between the synthesized first image and second image. To find theoptimal seam, first, defining the first image and second image into aplurality of points (step 442), each point has a residual error. Stitchthe two images along the optimal seam, and then get a third image 300(step 444). After the steps stated above, the partial scene images ofthe periphery of the vehicle can be synthesized into a completesurrounding view image. Suppose a 3D object presents on the seam 221,the edges of the objects in the first image will misalign to the edgesin the second image. In view of this problem, the method furthercomprises image registration 446, to align the edges of the object indifferent images. In this embodiment, dynamic image warping (DIW) hasbeen adopted. The method further comprises deforming the first image andthe second image by propagating the alignment on the seam to the rest ofthe first image and the second image and outputting a complete thirdimage (step 448). Finally, using weighted blending to compensate theexposure difference.

The present invention has already been described in details through theembodiments and the accompanying drawings. However, those skilled in theart should know that these embodiments are for illustration purposeonly, and are not meant to limit the present invention. Anymodifications or changes made to the embodiment are within the scope andspirit of the present invention. The present invention is set forth inthe attached claims.

1. An image synthesis system for a vehicle comprising: a first cameradisposed on the vehicle and shooting a first image of the periphery ofsaid vehicle; a second camera disposed on the vehicle and shooting asecond image of the periphery of said vehicle, wherein the second imageand the first image have an overlap region; an image processing devicecomprising: a defining component for defining the first image into aplurality of first points each has a first residual error, and definingthe second image into a plurality of second points each has a secondresidual error; and a synthesis component for synthesizing the firstimage and the second image and output a third image, wherein when thefirst point overlaps to the second point only the point which has lowerresidual error will be shown in the third image; and a display devicefor displaying the third image.
 2. According to the system as claimed inclaim 1, wherein the third image has at least a seam between thesynthesized first image and second image.
 3. According to the system asclaimed in claim 2, wherein at least one object presents on the seam andthe edges of the object in the first image misaligned to the edges inthe second image, and the image processing device further comprising aseam registration component for aligning the edges of the object on theseam.
 4. According to the system as claimed in claim 3, the imageprocessing device further comprising a deformation component fordeforming the first image and the second image by propagating thealignment of seam to the rest of the first image and the second image.5. According to the system as claimed in claim 1, wherein the imageprocessing device further comprising a transformation component fortransforming the first image and second image into a downward-facingimage.
 6. An image synthesis method for a vehicle, the methodcomprising: shooting a first image of the periphery of said vehicle by afirst camera disposed on the vehicle; shooting a second image of theperiphery of said vehicle by a second camera disposed on the vehicle,wherein the second image and the first image have an overlap region;processing the image comprising: defining the first image into aplurality of first points each has a first residual error, and definingthe second image into a plurality of second points each has a secondresidual error; and synthesizing the first image and the second imageand output a third image, wherein when the first point overlaps to thesecond point only the point which has lower residual error will be shownin the third image; and displaying the third image.
 7. According to themethod as claimed in claim 6, wherein the third image has at least aseam between the synthesized first image and second image.
 8. Accordingto the method as claimed in claim 7, wherein at least one objectpresents on the seam and the edges of the object in the first imagemisaligned to the edges in the second image, and the method furthercomprising aligning the edges of the object on the seam.
 9. According tothe method as claimed in claim 8, the method further comprisingdeforming the first image and the second image by propagating thealignment of seam to the rest of the first image and the second image.10. According to the method as claimed in claim 6, further comprisingtransforming the first image and second image into a downward-facingimage.
 11. An image synthesis device for synthesizing a first image anda second image, wherein the first image is taken by a first cameradisposed on a vehicle, and the second image is taken by a second cameradisposed on the vehicle, and the second image and the first image havean overlap region, the image synthesis device comprising: a definingcomponent for defining the first image into a plurality of first pointseach has a first residual error, and defining the second image into aplurality of second points each has a second residual error; and asynthesis component for synthesizing the first image and the secondimage and output a third image, wherein when the first point overlaps tothe second point only the point which has lower residual error will beshown in the third image.
 12. According to the device as claimed inclaim 11, wherein the third image has at least a seam between thesynthesized first image and second image.
 13. According to the device asclaimed in claim 12, wherein at least one object presents on the seamand the edges of the object in the first image misaligned to the edgesin the second image, and the device further comprising a seamregistration component for aligning the edges of the object on the seam.14. According to the device as claimed in claim 13, further comprising adeformation component for deforming the first image and the second imageby propagating the alignment of seam to the rest of the first image andthe second image.
 15. According to the device as claimed in claim 11,further comprising a transformation component for transforming the firstimage and second image into a downward-facing image.
 16. An imagesynthesis method for synthesizing a first image and a second image,wherein the first image is taken by a first camera disposed on avehicle, and the second image is taken by a second camera disposed onthe vehicle, and the second image and the first image have an overlapregion, the image synthesis method comprising: defining the first imageinto a plurality of first points each has a first residual error, anddefining the second image into a plurality of second points each has asecond residual error; and synthesizing the first image and the secondimage and output a third image, wherein when the first point overlaps tothe second point only the point which has lower residual error will beshown in the third image.
 17. According to the method as claimed inclaim 16, wherein the third image has at least a seam between thesynthesized first image and second image.
 18. According to the method asclaimed in claim 17, wherein at least one object presents on the seamand the edges of the object in the first image misaligned to the edgesin the second image, and the method further comprising aligning theedges of the object on the seam.
 19. According to the method as claimedin claim 18, further comprising deforming the first image and the secondimage by propagating the alignment of seam to the rest of the firstimage and the second image.
 20. According to the method as claimed inclaim 16, further comprising a transforming the first image and secondimage into a downward-facing image.